The present invention relates to photography, image processing and animation, and more particularly, but not exclusively to three dimensional (3D) photography, three dimensional image processing and three dimensional animation.
The present art in three-dimensional photography is based on the time dimension.
The present invention relates to several different fields that belong to the world of 3D imagery and image processing, for example: Stereoscopic images, spherical photographing systems, 3D computer animation, 3D photography, and 3D image processing algorithms.
Conventional 3D-stereoscopic photographing employs twin cameras having parallel optical axes and a fixed distance between their aligned lenses. These twin cameras produce a pair of images which can be displayed by any of the known in the art techniques for stereoscopic displaying and viewing. These techniques are based, in general, on the principle that the image taken by a right lens is displayed to the right eye of a viewer and the image taken by the left lens is displayed to the left eye of the viewer.
For example, U.S. Pat. No. 6,906,687, assigned to Texas Instruments Incorporated, entitled “Digital formatter for 3-dimensional display applications” discloses a 3D digital projection display that uses a quadruple memory buffer to store and read processed video data for both right-eye and left-eye display. With this formatter video data is processed at a 48-frame/sec rate and readout twice (repeated) to provide a flash rate of 96 (up to 120) frames/sec, which is above the display flicker threshold. The data is then synchronized with a headset or goggles with the right-eye and left-eye frames being precisely out-of-phase to produce a perceived 3-D image.
Spherical or panoramic photographing is traditionally done either by a very wide-angle lens, such as a “fish-eye” lens, or by “stitching” together overlapping adjacent images to cover a wide field of vision, up to fully spherical fields of vision. The panoramic or spherical images obtained by using such techniques can be two dimensional images or stereoscopic images, giving to the viewer a perception of depth. These images can also be computed three dimensional (3D) images in terms of computing the distance of every pixel in the image from the camera using known in art methods such as triangulation methods.
For example, U.S. Pat. No. 6,833,843, assigned to Tempest Microsystems Incorporated, teaches an image acquisition and viewing system that employs a fish-eye lens and an imager such as, a charge coupled device (CCD), to obtain a wide angle image, e.g., an image of a hemispherical field of view.
Reference is also made to applicant's co-pending U.S. patent application Ser. No. 10/416,533 filed Nov. 28, 2001, the contents of which are hereby incorporated by reference. The application teaches an imaging system for obtaining full stereoscopic spherical images of the visual environment surrounding a viewer, 360 degrees both horizontally and vertically. Displaying the images by means suitable for stereoscopic displaying, gives the viewers the ability to look everywhere around them, as well as up and down, while having stereoscopic depth perception of the displayed images. The disclosure teaches an array of cameras, wherein the lenses of the cameras are situated on a curved surface, pointing out from C common centers of said curved surface. The captured images are arranged and processed to create sets of stereoscopic image pairs, wherein one image of each pair is designated for the observer's right eye and the second image for his left eye, thus creating a three dimensional perception.
3D computer animation relates to the field of “Virtual Reality”, that has gained popularity in recent years. 3D Virtual reality is constructed from real images, with which synthetically made images can be interlaced in. There also exists fully computer generated Virtual reality. 3D virtual reality demands 3D computation of the photographed image to create the 3D information of the elements being shot.
This can be done in real time using active methods.
For example, 3DV systems Incorporated provides the ZCam™ camera which captures, in real time, the depth value of each pixel in the scene in addition to the color value, thus creating a depth map for every frame of the scene by grey level scaling of the distances. The Zcam™ camera is a uniquely designed camera which employs a light wall having a proper width. The light wall may be generated, for example, as a square laser pulse. As the light wall hits objects in a photographed scene it is reflected towards the ZCam™ camera carrying an imprint of the objects. The imprint carries all the information required for the reconstruction of the depth map.
3D computation of photographed images may also be provided using passive methods.
Passive methods for depth construction may use triangulation techniques that make use of at least two known scene viewpoints. Corresponding features are identified, and rays are intersected to find the 3D position of each feature. Space-time stereo adds a temporal dimension to the neighborhoods used in the spatial matching function. Adding temporal stereo, using multiple frames across time, we match a single pixel from the first image against the second image. This can also be done by matching space-time trajectories of moving objects, in contrast to matching interest points (corners), as done in regular feature-based image-to-image matching techniques. The sequences are matched in space and time by enforcing consistent matching of all points along corresponding space-time trajectories, also obtaining sub-frame temporal correspondence (synchronization) between two video sequences.
3D computer generated images (CGI) is a virtual world, a designated area, created using 3D computer generated images software. The virtual world is created in a designated area where every point in the virtual world is a computer generated point. 2D or 3D real images may also be interlaced in this virtual world.
Reference is now made to FIG. 1 which illustrates a virtual world, according to techniques known in the art.
The 3D position of every point in this virtual world is known. Adding to certain points in the space details such as color, brightness and so on, creates shapes in space (FIG. 1). Introducing a virtual camera into this world enables to create time based sequences in the virtual world, to create stereo images, and so on.
We can synchronize between photographed images and the computer generated world using space synchronization, and then time synchronization, fitting real world images in the virtual world in spatial and temporal terms.
Reference is now made to FIG. 2 which shows a prior art virtual studio.
In this example we use a virtual studio where the camera enables to create separation between a human figure and its background, in a technique which is known in art as blue/green screen. Isolating the human figure from its surrounding we can interlace the figure in the virtual world created in a computer, as shown in FIG. 3.
The very opposite thing can also be done by monitoring a set of cameras in a pre-determined space such as a basketball field, where known fixed points are pre determined, and synchronized fix points are created in a computer generated 3D world. With such a technique, we can isolate a CGI figure and interlace it in the basketball field. For example, ORAD Incorporated CyberSport™ product provides for live insertion of tied-to-the-field 3D graphics for sport events taking place in a basketball field, a football field, and the like, creating the illusion that the inserted graphic objects are integral parts of the event.
As described above, traditional methods and systems for 3D imaging and stereoscopic photography are based on special cameras, special lenses, predetermined positioning of two or more cameras and dedicated algorithms.
There is thus a widely recognized need for, and it would be highly advantageous to have a system and method for photography and imaging